The Problem of Neural Networks in Business Forecasting:An Attempt to Reproduce the Hill, Oâ€TMConnor and Remus Study

The results from applying neural networks to business forecasting have been mixed. Among the most encouraging efforts is that of Hill, O’Connor and Remus (1996). In that study, neural networks produced forecasts that were significantly better than those produced by traditional methods for quarterly and monthly series, and no worse for annual series. We have attempted to reproduce that study. The pattern of our results matches that of the original study and supports many of its conclusions, but we were not able to obtain nearly the magnitude of the improvements reported there. We observe that the stopping rules proposed in the original study have exceedingly high variance associated with them, and do not seem to be reasonable stopping rules for such applications. We conclude that the original study’s networks, and in particular its stopping rules, are not described in a way that permits reproducing them. This undermines the value of these systems in practice, since it is difficult both to reproduce them and to identify when and why they are failing to produce results like those obtained in research settings. At the same time, Hill et al.’s conclusion that neural networks represent a good approach to extrapolating nonlinear and discontinuous series is supported.